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Abstract. In this paper, we show that the low rank matrix completion problem can be 
reduced to the problem of finding the rank of a certain tensor. 

1. INTRODUCTION 

Suppose we are given a rectangular array which has only been partially filled out with 
entries in M (or some other field). The low rank matrix completion problem asks to fill out 
the remaining entries such that the resulting matrix has minimal rank. For example, the 
array 



can be completed to a rank 1 matrix 





but clearly cannot be completed to a rank matrix. For a survey on matrix completion 
problems, see [HI [12] . The low rank matrix completion problem has various applications, 
such as statistics, computer vision, signal processing and control. The low rank matrix 
completion problem is known to be NP-hard (see [3]). For the field F = M or F = C one can 
solve the matrix completion problem using convex relaxation: under certain assumptions 
minimizing the nuclear norm of the matrix (the sum of the singular values) one also obtains 
the minimal rank completion. See for example fl8\ HI [5l [15] . The rank of a tensor was defined 
in [11]. Using this notion, matrix completion problems can be generalized to higher order 
tensors (see [7]). 

Suppose that F is a field, and y'-*-' = F"*' for i = 1,2, ... ,d. The tensor product space 

V = ® ^ . . . yC'i) 

can be viewed as a d- dimensional array of size mi x m2 x ■ ■ ■ x m^. A pure tensor is an 
element of V of the form 

v'^^^ (g) V^'^^ (g) ■ ■ ■ ® v^'^\ 

where v^^^ E V^^^ for all i. The rank of a tensor T G is the smallest nonnegative integer r 
such that T can be written as a sum of r pure tensors: 



i=l 
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where vf^ G y*-*-* for all i and j. We denote the tensor rank of T by rank(T). \i A= (ajj) is 
an n X m matrix, then we can identify A with the tensor 

n m 

i=i j=i 

in F" ® F™, where Cj denotes the i-th basis vector. The tensor rank in this case is exactly 
the same as the rank of the matrix. For c? > 3 it is often difficult to find the rank of a 
given tensor. Finding the rank of a tensor is NP-complete if the field F is finite ([9l [TO] ) 
and NP-hard if the field is Q ([9l [10]), or if it contains Q ([E]). Approximation of a tensor 
by another tensors of low rank is known as the PARAFAC (0) or CANDECOMP (^2]) 
model. There are various algorithms for finding low-rank approximations. See [T^] for 
a discussion. The problems of finding the tensor rank of a tensor, and to approximate 
tensors with tensors of low rank has many applications, such as the complexity of matrix 
multiplication, fiuorescence spectroscopy, statistics, psychometrics, geophysics and magnetic 
resonance imaging. In general, the rank of higher tensors more ill-behaved than the rank of 
a matrix (see |6J). 

In the next section we will show that the low rank matrix completion problem can be 
reduced to finding the rank of a certain tensor. Lek-Heng Lim pointed out to the author, 
that because low rank matrix completion is NP-hard (see [5|), this gives another proof that 
determining the rank of a tensor is NP-hard. 

2. The main result 

Suppose that F is a field, and A = (aij) is an n x m array for which some of the entries 
are given, and the entries in positions (ii,ii), . . . , {is,js) are undetermined. Define Cij as 
the matrix that has a 1 in position and everywhere else. We can reformulate the low 
rank matrix completion problem as follows. Define aj^j^. = for k = 1,2, ... ,s and find a 
matrix B of the form 

s 

B = A + ^XkCi^j^, Ai,...,A, gF. 

k=l 

which has minimal rank. 

We can view matrices as second order tensors. So we identify Cij with Cj (8> cj G F" ® F*" 
and we have 

n m 

A = ^^aij{ei(^ej). 
i=i j=i 

Define Uk = and = ej^. The matrix completion problem asks to find Ai, . . . , G F 
such that the tensor 

s n m s 

B = A + ^\k{uk ® Vk) = ttijici (g) ej) + ^ Xkiuk ® ffc) 

k=l i=l j=l k=l 

has minimal rank. 

Let C F"- ® F™ be the vector space spanned by ui ^ vi, ... ,Us ^ Vs. 

Definition 1. We define 

rank(A, U) = min{rank(5) | B E A + U}. 
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So the matrix completion asks for the value of rank(74, U) and to find a matrix B E A + U 
for which rank(A, U) = rank(i?). 

We define a third order tensor A G F" F™ (g) F'*^^ by 



s n m s 



(1) A = es+i + ^Mfc ® ffc ® Cfc = ^^aij{ei ® ej ® e^+i) + ® Wfc ® e^. 

fc=l i=l j=l k=l 



Our main result is: 
Theorem 2. 1/Ke /lawe 



rank(A, U) + s = rank(74). 



The theorem is effective: given an explicit decomposition of A as a sum of / = rank(y4) 
pure tensors, the proof shows that one easily can construct a matrix B in A + U such that 
rank(5) = I - s = rank(v4, U). 

Proof of Theorem\^ Let r = rank(y4, ?7) and / = rank(y4). By definition, there exists a 
matrix B E A + U with rank r. We can write 

s 

B = A + ^ \k{uk ® Vk). 

k=l 

Since B has rank r, we also can write 



Xi Vi 



1=1 



for some Xi, . . . ,Xr G F" and yi, . . . ,yr G F™. We have 

s s s 

A = A^ Cs+i + ^ Mfe ® Vfc ® Cfc = (5 - ^ AfcMfc (g) Vfc) (g) e^+i + ^ Mfe ® ^^fc ® = 
fc=l fe=l fc=l 

= S (g) e^+i + ^Uk^Vk^ {ek - AfcC^+i) = ^ (g) j/j + ^ (g) ® (e^ - AfcC^+i). 

fc=l i=l k=l 

We have written A as a sum of r + s pure tensors. So I = rank(yl) < r + s. 
Conversely, we can write 

I 

A = ^ Oj (g bj (g Cj 
i=i 

For every i, choose a linear function hi : F" (g F™ — )■ F such that hi{ui (g Vj) = 1 and 
hi{uj (g i;^) = if j 7^ i Applying hi ^ id: F" (g F"" (g F'*+^ F*+^ to the tensor A gives 

/ii(aj (g f)j)cj = (/ij (g id)(A) = Ci + /ii(74)es+i. 

This shows that lies in the span of Ci, . . . , q, e^+i for all i. So Ci, . . . , q, e^+i span the 
vector space F'^"'"-'^. After rearranging ci, . . . , q, we may assume that Ci, C2, . . . , c^, e^+i is a 
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basis of F*"*"^. There exists a linear function / : F*+^ — t- F such that /(ci) = ■ ■ ■ = f{cs) = 
and /(cs+i) = 1. We apply id ® id ®/ : F" ® F*" O ¥'+^ F" O F™ to A: 

s I I 

A + Y^ f{ej)ui ®Vi = (id ® id ®f){A) = ^ f{cj)aj ®hj= ^ /(cj)aj- ® hj. 

i=l j=l j=s+l 

The tensor above is a matrix in A + U which has rank at most / — s, and by definition of 
r = rank(yl, U) it has rank at least r. It follows that r <l — s. We have already proven that 
r + s > Z, so we conclude that r + s = I. □ 

Remark 3. In the proof we have not used that Uk and Vk are basis vectors. So Theorem [2] 
is true whenever U is the span of linearly independent pure tensors 

and A is given by ([Tj). 
Remark 4. Define 

ti e (¥^)^' = F^ ® ■ ■ ■ (g) F^ 



by 

U = ei ® ■ ■ ■ ® ei ® 62 ® ei ® ■ ■ ■ ® d 
where 62 appears in the i-th position if z < s, and 

ts+i = ei ® ei ® ■ ■ ■ ® ei. 

We define a tensor A G F" ® F™ O (F^)®" by 

A = Ml (g) -Ui (g) ti + ■ ■ ■ + Ms (g) t;^ (g) + y4 (g) ts+i. 

A proof, similar to that of Theorem [2] shows that rank(yl) = s + rank(yl, U). The ambient 
vector space of A is much larger than that of A. But one can imagine that using the tensor 
A can be advantageous for proving lower bounds for rank(74, U) because it is, in a way, more 
rigid. 

Remark 5. Theorem [2] also easily generalizes to the higher order analog of matrix comple- 
tion: tensor completion. Suppose that V^^^ , • • • , V^^^ are finite dimensional F- vector spaces 
and V = V^^^ ® ■■■ ® V'-'^\ Assume that U C V is a subspace spanned by s linearly 
independent pure tensors 

u[^'^ ®uf^ ®---®uf, i = l,2,...,s 

and 

A G 1/^^) ® ■ ■ ■ ® 

is a tensor. As before, we define 

rank(yl, U) = min{rank(5) \ B e A + U}. 
Define a. {d + l)-th order tensor by 

s 

A = A® e,+i + ^4'^ ® ■ ■ ■ ® ® G V^^^ ®---® V^"^^ ® ¥'+\ 

k=l 



Then we have 



rank(A, [/) + s = rank(^). 
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